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A general condition determining the optimal performance of a complex system has not yet been 
found and the possibility of its existence is unknown. To contribute in this direction, an optimization 
algorithm as a complex system is presented. The performance of the algorithm for any problem is 
controlled as a convex function with a single optimum. To characterize the performance optimums, 
certain quantities of the algorithm and the problem are suggested and interpreted as their complex- 
ities. An optimality condition of the algorithm is computationally found: if the algorithm shows 
its best performance for a problem, then the complexity of the algorithm is in a linear relationship 
with the complexity of the problem. The optimality condition provides a new perspective to the 
subject by recognizing that the relationship between certain quantities of the complex system and 
the problem may determine the optimal performance. 



PACS numbers: 89.75.-k, 89.75.Fb 

The efficient management of complex systems is 
increasingly important. However, despite significant 
progress and interest in complex systems [lj, there is 
a limited understanding of the problem. In particular, 
because the existence of principles governing the non- 
equilibrium situation has not yet been established , the 
possibility of a general condition determining the optimal 
performance of a complex system is still unknown. We 
aim to contribute in this direction and present results of 
computational experiments in order to discuss the possi- 
bility of an optimality condition of complex systems. The 
results provide a new perspective to the subject by rec- 
ognizing that the relationship between certain quantities 
of the complex system and the problem may determine 
the optimal performance. 

In this Letter, we consider an optimization algorithm 
as a complex system by using the benchmark traveling 
salesman problems 0- For any problem tested, we can 
control the performance of the algorithm as a convex 
function with a single optimum. Consequently, we take 
this opportunity to investigate whether the performance 
optimums may be characterized in terms of an optimal- 
ity condition of the algorithm. Namely, we describe the 
algorithm by a trace of the variance-covariance matrix 
derived from the dynamics of the algorithm as a complex 
system. We also specify a problem by a trace of the dis- 
tance matrix. Remarkably, the computational analysis 
of the performance optimums reveals a relationship be- 
tween the quantities of the algorithm and the problem 
by approximating it well enough with a linear function. 

To recognize the potential importance of the result, 
we interpret the quantities as the complexities of the al- 
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gorithm and the problem, and formulate the following 
optimality condition: if the algorithm shows its best per- 
formance for a problem, then the complexity of the al- 
gorithm is in the linear relationship with the complexity 
of the problem. Further experimental facts suggest how 
the optimality condition may be extended into a wider 
context. 

Let us consider an optimization algorithm A as a com- 
plex system of N computational agents that minimize the 
average distance in the traveling salesman problem. All 
agents start in the same city and at each step an agent 
visits the next city by using one of two strategies: a ran- 
dom strategy, i.e., visit the next city at random, or the 
greedy strategy, i.e, visit the next closest city. We de- 
fine that all agents start with the random strategy. The 
state of the agents visiting n cities of the problem can be 
described at step j = 1, n — 1 by a binary sequence 
Sj = Sij...SNj, where Sij = +1, if agent i = 1, ...,N uses 
the random strategy and = —1, if the agent uses the 
greedy strategy to visit the next city. The dynamics of 
the agents is realized through their choice of strategies 
and can be encoded by an JV x (n — 1) binary strategy 



matrix S = {sj 



1,...,N, j = l, 



!}■ 



We introduce a variable parameter v that controls the 
dynamics of the agents in a specific manner. Let Dij 
be the distance travelled by agent i = 1, N after j = 
1 , . .. , n — 1 steps and 



DJ = min D ijt D+ 

3 i=l,...,N J 3 



max Di 

i=l,...,JV 



All distances travelled by the agents after j = 1, n — 1 
steps belong to the interval [Dj,D^]. The parameter v 
specifies a threshold point 

D j (v)=Df -v{Dj -DJ), 0<u<l + 7, 7 = 0.1 

dividing the interval into two parts, i.e., successful 
[DJ ,Dj(v)\ and unsuccessful (Dj(v), D^]. If the dis- 
tance D^ travelled by agent i = 1,...,N after j — 
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1, ...,n — 1 steps belongs to the successful interval, then 
we regard the agent's last strategy as successful. If the 
distance Dij belongs to the unsuccessful interval, then 
the agent's last strategy is unsuccessful. 

In order to choose the next strategy, each agent uses 
an optimal rule £| that relies on the Prouhet-Thue-Morse 
(PTM) sequence and has the following description: 

1. If your last strategy is successful, continue with the 
same strategy. 

2. If your last strategy is unsuccessful, consult PTM 
generator which strategy to use next. 

The PTM sequence +1-1-1 + 1-1 + 1 + 1-1 ... gives 
a symbolic description of chaos resulting from period- 
doubling p| in complex systems @|. It can be also asso- 
ciated with formation processes of integer relations pro- 
gressing well through the levels of a hierarchical struc- 
ture The formation processes produce a measure of 
complexity Q that we use as a guide in defining the com- 
plexities of the algorithm and the problem. 

Each agent has its own PTM generator and a pointer 
attached to it. The pointer starts with the first bit of the 
PTM sequence and after each consultation moves one 
step further, so that the next bit of the PTM sequence 
can be used, if the strategy is unsuccessful. The control 
is realized by changing the parameter v from to 1 + 7. 
For the limiting values of the parameter v, the algorithm 
A produces the following dynamics of the agents. 

When v = 0, then Dj(v) = Dj. Therefore, the suc- 
cessful interval [Dj , Dj(v)] coincides with the whole in- 
terval [D~ ,D~j~] and the last strategy of each agent is suc- 
cessful at each step j — 1, .. . , n — 1 . This means that each 
agent always uses the random strategy and the strategy 
matrix becomes 
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In the opposite limit, when v — 1 + 7, then Dj(v) < 
DJ . Thus, the unsuccessful interval (Dj(v),Dj] covers 
the whole interval [D~,D^]. This means that the last 
strategy of each agent is always unsuccessful and, ac- 
cording to the rule, at each step j = 1, n — 1 an agent 
asks the PTM generator which strategy to use next. As 
a result, the binary sequence of each agent becomes the 
initial segment of length (n — 1) of the PTM sequence 
and the strategy matrix is turned to 
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The role of the parameter v may be seen in terms of 
correlations. When v — 0, the agents are independent, 
because the behavior of an agent does not depend on 
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FIG. 1: The performance of the algorithm A as a convex 
function of the parameter v for problem eilOl. 



the others. However, as the parameter v gets larger, 
the successful interval gets smaller, and the behaviors 
of the agents become more correlated through the PTM 
sequence. To stay independent, an agent has to show a 
result approaching the current minimum. Consequently, 
the rule urges an agent to follow the PTM sequence more 
strongly and as a result, the presence of the PTM se- 
quence in the strategy matrix becomes more evident. 
When v = 1 + 7, the whole strategy matrix consists of 
the PTM sequences. 

By extensive computational experiments, we investi- 
gate how the performance of the algorithm A may depend 
on the parameter v. We tested the problems P = {eil76, 
eillOl, st 70, ratl95, linl05, kroClOO, kroBlOO, kroAlOO, 
kroDlOO, dl98, kroAl50, prl07, ul59, prl44, prl44, 
prl52, pr226, prl36, pr76, ts225} belonging to the bench- 
mark traveling salesman problems |3(. Let Di(p,v) be 
the distance travelled by agent i = 1, ...,N for a problem 
p G P and a value v of the parameter. The performance 
of the algorithm A is characterized by the average dis- 
tance travelled by N agents D(p,v) — Yli=i Di(p,v)/N, 
which is sought to be minimized. 

In the experiments, the algorithm A is applied to each 
problem pGP with the number of agents N — 2000 and 
values Vi — iAv, i = 0, 1, ...,20, v = 1.1 of the param- 
eter, where Aw = 0.05. To eliminate randomness, the 
computations are repeated in a series of 10 tests and the 
performance functions {Dk(p, v), k = 1, 10} are aver- 
aged into D(p, v). Remarkably, for each problem p G P, 
it is found that the performance D{p, v) of the optimiza- 
tion algorithm A behaves as a convex function of the 
parameter v with the only optimum at v*(p) (Fig. 1). 

We characterize the performance optimums 
{v*(p), p G P} by certain quantities of the algo- 
rithm and the problem and interpret the quantities as 
their complexities. The algorithm A is described by 
the quadratic trace of the variance-covariance matrix 
derived from the strategy matrix 0. Namely, for a 
problem p G P, a set of strategy matrices Sk(v*(p)) = 
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{sij(A),i = h-,N; j = l,...,n-l},fe = 1,...,10 
is obtained as a result of ten runs for the value 
v*(p) of the parameter. For each strategy matrix 
Sk(v*(p)),k — 1, 10 the variance-covariance matrix 
V(S k (v*(p))) = {Vij(k), i,j = l,...,N},i = 1,...,10, 
and its quadratic trace 

N 

tr(V 2 (S k (v*(p)))) = J2^ k 

i=l 

are computed, where Vij(k) is the linear correlation co- 
efficient between agents i = 1, N and j = 1,...,N and 
Xik,i = 1, —,N, k = 1, 10 are the eigenvalues of the 
variance-covariance matrix V(Sk{v* (p))). 

The average tr(V 2 (S(v*(p)))) of the traces is used to 
describe the complexity 

C(A(p))^^tr(V 2 (S(v*(p)))) 

of the algorithm A. There is a connection between 
N 2 /tr(V 2 (S)) and the number of KLD (Karhunen-Loeve 
decomposition) modes Dkld |8|- The quantity Dkld 
measures the complexity of spatiotemporal data and 
a correlation length ^kld-, based on Dkld, can char- 
acterize high-dimensional inhomogeneous spatiotemporal 
chaos |l0j . In our case, it turns more appropriate to 
consider the complexity of the algorithm A in terms of 
tr(V 2 (S))/N 2 , although the complexity is greater, the 
smaller tr{V 2 (S))/N 2 . 

We describe the complexity C (p) of the problem p G P 
by the quadratic trace 

1 1 n 

C (p)= tr(M 2 (p)) = 

i=i 

of the distance matrix M(p) — {dij / d max ,i, j = 1, ...,n}, 
where Xi,i — l,...,n are the eigenvalues of the distance 
matrix, is the distance between cities i = 1, ...,n and 
j = 1, n and d m az is the maximum of the distances. 

An optimality condition of the algorithm A is sought 
through a possible relationship between the complexity 
C(A(p)) of the algorithm A and the complexity C(p) of 
the problem p. For this purpose, we consider the points 
with coordinates {x = C(p),y = C(A(p)), p G P}. The 
result of the analysis shown in Fig. 2 suggests a pos- 
sible linear relationship between the complexities. The 
regression line is calculated 

y = ax + = 0.67x + 0.33, 



a = 0.67 ±0.01, = 0.33 ±0.01, (1) 

where the standard error of estimate is 0.09 and the ab- 
solute value of the maximal individual error is 0.05. The 
coefficient of determination of 0.71 tells that 71 percent 
of the variation in the complexity of the algorithm A is 
explained by the regression line. 
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FIG. 2: The regression line of the relationship between the 
complexity of the algorithm A and the complexity of the prob- 
lem with the bounds including all points. 

The experiment is repeated a number of times and each 
time the results confirm the consistency of the regression 
line JTJ. Therefore, within the accuracy of the linear 
regression, we are able to formulate an optimality condi- 
tion: if the algorithm A shows its best performance for a 
problem p, then the complexity C(A(p)) of the algorithm 
A is in the linear relationship with the complexity C(p) 
of the problem p 

C{A{p)) = 0.67 x C{p) + 0.33. (2) 

Computational investigations on the role of the PTM 
generator for the algorithm A suggest how the optimality 
condition J2J may be extended. For this reason we use 
a different algorithm B, which works exactly in the same 
manner as the algorithm A, except it consults a random 
generator instead of the PTM generator. To find a pos- 
sible connection between the best performances and the 
relationship between the complexities, we compare the 
algorithms A and B. 

First, for each problem p G V the best performance of 
the algorithm A is compared with the best performance 
of the algorithm B. Fig. 3 shows that the algorithm A 
demonstrates significantly better results than the algo- 
rithm B for seventeen problems and results for the other 
three are close. 

Second, we examine the relationship between the com- 
plexities of the algorithm B and the problem (Fig. 4) in 
the same manner as we have done for the algorithm A 
(Fig. 2). The situation deteriorates and the relationship 
is less consistent for the algorithm B in comparison with 
the algorithm A. Formally, the calculated regression line 

y = a'x + P' = 0.48a; + 0.23, 

a' = 0.48 ±0.02, /3' = 0.23 ± 0.01 (3) 

is a less accurate estimator of the relationship between 
the complexities of the algorithm B and the problem. 
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FIG. 3: The best performances of the algorithm A using 
PTM generator are compared with the best performances of 
the algorithm B using random generator. The problems are 
ordered as in the description of the class P. 



The standard error of estimate of 0.18 and the absolute 
value of the maximal individual error of 0.1 of the re- 
gression line @ are doubled in comparison with the re- 
gression line ■ The coefficient of determination is only 
0.26 in comparison with 0.71 for the algorithm A. 

Therefore, computationally, we reveal a connection be- 
tween the best performances and the relationship be- 
tween the complexities. In particular, the better result of 
the algorithm A (Fig. 3) corresponds to the fact that the 
relationship between the complexities of the algorithm A 
and the problem appears more consistent than it is in 
the case of the algorithm B. In order to suggest poten- 
tial implications of the optimality condition J2} to a wider 
context, we would like to use this proposition. 

The observed connection allows us to assume a possi- 
ble sequence of algorithms whose best performances con- 
verge to the optimal solutions of the problems as the 
relationships between the complexities converge to a cer- 
tain function g. Notice, that the algorithms A and B 
may be well among the elements of the sequence with 
the algorithm B followed by the algorithm A. According 
to the experiments, it seems likely that the function g 
may be a linear function. 

The possible convergence of the sequence would 
promise an algorithm Q to link the optimal performance 
and the relationship between the complexities. As a re- 
sult, the best performance of the algorithm Q for any 
problem could actually provide the optimal solution. An 
optimality condition of the algorithm Q may extend the 
optimality condition of the algorithm A and explain the 
optimal performance in terms of the relationship between 
the complexities: if the algorithm Q shows its best per- 
formance for a problem p, and thus finds the optimum 
solution, then the complexity C(Q(p)) of the algorithm 
is in the relationship C(Q(p)) = g(C(p)) with the com- 
plexity C [p) of the problem. 

The optimality condition of the algorithm Q may sug- 
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FIG. 4: The regression line of the relationship between the 
complexity of the algorithm B and the complexity of the prob- 
lem. 



gest a new approach to optimization that would be pri- 
marily concerned with the efficient control of the algo- 
rithm's complexity in matching the complexity of the 
problem. For example, if the algorithm Q could poten- 
tially find the optimal solution to a problem p of complex- 
ity C(p) under the condition C(Q(p)) = g(C(p)), would 
it then be possible to obtain the optimal solution by a 
different algorithm working with the same complexity for 
the problem? 

The ability to evaluate the complexity of the prob- 
lem before the actual computation would be beneficial. 
In this case, the finding of the optimal solution could 
be connected with the tuning of the algorithm's com- 
plexity in order to match the already known complexity 
of the problem. Experimentally, it is observed that for 
any problem the performance of the algorithm A is a 
convex function of the parameter. As a result, the best 
performance of the algorithm A for the problem can be 
efficiently obtained by the minimization of this convex 
function. Therefore, it would be important to under- 
stand whether the performance of the algorithm Q, as 
the function of its complexity, may behave in a similar 
way to provide us with efficient means for finding optimal 
solutions. 

In conclusion, we have presented an optimization al- 
gorithm as a complex system. For any problem tested, 
the performance of the algorithm has been controlled as 
a convex function with a single optimum. By the char- 
acterization of the performance optimums, an optimality 
condition of the algorithm has been proposed. The op- 
timality condition has revealed that the relationship be- 
tween certain quantities of the complex system and the 
problem may determine the optimal performance. The 
result provides a computational evidence to the possibil- 
ity of an optimality condition of complex systems. 

This work was supported by CQU Research Advance- 
ment Awards Scheme grant no. IN9022 and Faculty of 
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no. FRG00406. 
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